Adaptive Learning in Innite Horizon Decision Problems
نویسنده
چکیده
Building on Marcet and Sargent (1989) and Preston (2005) this paper shows that for in nite horizon decision problems in which agents optimize but have arbitrary subjective expectations about the evolution of state variables beyond their control optimal decision rules necessarily depend on in nite-horizon expectations. This contrasts with most work on adaptive learning in macroeconomics since Marcet and Sargent (1989) which posit Euler equations as decision rules in which only one-period-ahead forecasts matter. Using the Townsend (1983) investment model and the canonical consumption model various pathologies of the Euler equation approach are adduced. The two modeling approaches are shown to give very di¤erent conclusions about policy in a simple model of output gap and ination determination.
منابع مشابه
Finite Horizon Learning
Incorporating adaptive learning into macroeconomics requires assumptions about how agents incorporate their forecasts into their decision-making. We develop a theory of bounded rationality that we call finite-horizon learning. This approach generalizes the two existing benchmarks in the literature: Eulerequation learning, which assumes that consumption decisions are made to satisfy the one-step...
متن کاملOptimal Adaptive Learning in Uncontrolled Restless Bandit Problems
In this paper we consider the problem of learning the optimal policy for uncontrolled restless bandit problems. In an uncontrolled restless bandit problem, there is a finite set of arms, each of which when pulled yields a positive reward. There is a player who sequentially selects one of the arms at each time step. The goal of the player is to maximize its undiscounted reward over a time horizo...
متن کاملSolving Markov Decision Processes via Simulation
This chapter presents an overview of simulation-based techniques useful for solving Markov decision problems/processes (MDPs). MDPs are problems of sequential decision-making in which decisions made in each state collectively affect the trajectory of the states visited by the system — over a time horizon of interest to the analyst. The trajectory in turn, usually, affects the performance of the...
متن کاملFast Planning in Stochastic Games
Stochastic games generalize Markov decision processes (MDPs) to a multiagent setting by allowing the state transitions to depend jointly on all player actions, and having rewards determined by multiplayer matrix games at each state. We consider the problem of computing Nash equilibria in stochastic games, the analogue of planning in MDPs. We begin by providing a generalization of nite-horizon v...
متن کاملA Learning Rate Analysis of Reinforcement Learning Algorithms in Finite-Horizon
Many reinforcement learning algorithms, like Q-Learning or R-Learning, correspond to adaptative methods for solving Markovian decision problems in innnite-horizon when no model is available. In this article we consider the particular framework of non-stationary nite-horizon Markov Decision Processes. After establishing a relationship between the nite-horizon total reward criterion and the avera...
متن کامل